Abnormal pattern detection processing method and system

ABSTRACT

The result of detection processing by abnormal pattern detection processor means 30 is stored in a server 40. In addition, the output image on an image output device 440 is read for pattern, and the supporting contents, such as the result of judgment by a doctor (which, when differing from the result of detection processing, corresponds to the corrected result), the comment, and the assessment category, are stored in the server 40 in conformity with the BI-RADS proposed by the ACR. The result of pathologic assessment corresponding to the abnormal pattern is related to the supporting contents and stored in the server 40. The statistical processing in conformity with the follow-up method of the BI-RADS is performed to find the specificity, the cancer detection rate, etc.

This is a continuation of application Ser. No. 09/489,846 filed Jan. 24,2000. The entire disclosure of the prior application, application Ser.No. 09/489,846 is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an abnormal pattern detectionprocessing method and system, and particularly, to a computer-aided typeof abnormal pattern detection processing method and abnormal patterndetection processing system which, on the basis of image datarepresenting a radiation image of a subject, detects and processes anabnormal pattern in the radiation image, and, when it is determined thatan abnormal pattern exists, displays an image of the affected part zoneincluding this abnormal pattern to supply it for assessment.

2. Description of the Prior Art

Up to now, reading a radiation image of a subject recorded on anaccumulative fluorescent material sheet or film to obtain image data,and after providing an appropriate image process for this image data,generating the image by use of a display device, etc., have beenperformed in a variety of fields, such as the medical field.Particularly, in recent years, digital image processing technology hasbeen developed in combination with computers, and the CT (ComputedTomology) device, the MRI (Magnetic Resonance Imaging) device, the CR(Computed Radiography) device and various other image forming modalities(image input devices) have been in widespread use as devices for formingimages for assessment, etc.

With the popularization and advance of network technology, in themedical field, for example, it is being realized to construct a medicalimage network for assessment with which a variety of image formingmodalities installed in the audit room, etc., in a hospital areconnected by network to image output devices, such as an image displaydevice (such as a CRT and a liquid crystal display) and a printer(including a laser printer (LP)), installed in a medical examination andtreatment room, a research laboratory, etc., which allows imageinformation for assessment acquired with an image forming modality inthe audit room to be obtained in the medical examination and treatmentroom without leaving it. Further, it has been proposed to provide, onsuch a medical image network for assessment, a workstation for qualityassurance (hereafter called QAWS (Quality Assurance Workstation)) whichcollectively control the image information acquired with theabove-mentioned various image forming modalities, the image informationobtained by providing a variety of image processes for this imageinformation to improve the assessment performance, etc.

Incidentally, the above-stated digital image processing technology hasfeatures which are essentially different from those of the conventionalanalog type in that it can quantitatively analyze the image data. Forexample, the abnormal pattern detection processing technology known ascomputer-aided image assessment or CADM (Computer Aided Diagnosis ofMedical Image), which is intended to more positively utilize thefeatures of this digital image processing technology for medicalassessment of human bodies, has been proposed (refer to “DetectionMethod of Malignant Tumors in DR Images —Iris Filter”, ElectronicsInformation Communication Society article magazine, D-II Vol. J75-D-JJNo. 3p. 663 to 670, March, 1992, “Extraction of Microcalcifications onMammogram Using Morphological Filter with Multiple StructuringElements”, the same magazine, D-II Vol. J75-D-II No. 7 p. 1170 to 1176,July, 1992, etc.).

This abnormal pattern detection processing technology uses a computer todetect, on the basis of the image data representing a radiation image,an abnormal tumor pattern indicating a cancer or the like, or anabnormal pattern candidate which can be considered as a high-densityminute calcification pattern or the like (hereafter these aregenerically called an abnormal pattern), and provides a marking for thedetected portion to attract the attention of a pattern reader (forexample, a radiologist) who observes the radiation image to read thepattern, or quantitatively offers the distinctive one of the abnormalpattern candidates detected as useful data for objective judgment of thepattern reader, thus preventing oversight due to the difference inpattern reading capability of the pattern reader, and misunderstandingdue to subjective judgment, thereby improving the assessmentperformance.

The present applicant has also proposed an abnormal pattern detectionprocessing system which is well suited for configuring abnormal patterndetection processing as a device independent of the QAWS andconstructing a network (Japanese Unexamined Patent Publication No. 10(1998) 233815). This abnormal pattern detection processing systemcomprises image selector means which selects, among the items of imageinformation which are inputted from an image input device equivalent tothe above-mentioned image forming modality, being provided withsupplementary information which allows identification of the type ofsubject and the patient[[,]]the image information concerning aparticular type of subject which is to be an object of abnormal patterndetection processing by the abnormal pattern detection processormeans[[,]]and outputs it, and input monitor means which, when an item ofimage information concerning a subject which is to be an object ofabnormal pattern detection processing is inputted from the imageselector means, monitors that all the other items of image informationconcerning the same subject for the same patient which are to provide aset with the item of image information concerning the subject which isto be an object are inputted from the image selector means[[,]]and, whenhaving detected that all the items of image information have beeninputted, causes collective inputting of all these items of abnormalpattern detection processing object image information concerning thesame subject for the same patient to the abnormal pattern detectionprocessor means. With this configuration, an automatic routing functionis provided which, from a number of and a variety of items of imageinformation inputted in the random order, automatically searches out andcollects the items of image information which provide a set for eachparticular patient, thus eliminating the need for manual operation bythe operator to select and output the items of image information to beoutputted to the abnormal pattern detection processor means.

The present applicant has also proposed an abnormal pattern detectionprocessing system which is well suited for configuring abnormal patterndetection processing as a device independent of the QAWS andconstructing a network (Japanese Unexamined Patent Publication No. 10(1998)-233815). This abnormal pattern detection processing systemcomprises image selector means which selects, among the items of imageinformation which are inputted from an image input device equivalent tothe above-mentioned image forming modality, being provided withsupplementary information which allows identification of the type ofsubject and the patient, the image information concerning a particulartype of subject which is to be an object of abnormal pattern detectionprocessing by the abnormal pattern detection processor means, andoutputs it, and input monitor means which, when an item of imageinformation concerning a subject which is to be an object of abnormalpattern detection processing is inputted from the image selector means,monitors that all the other items of image information concerning thesame subject for the same patient which are to provide a set with theitem of image information concerning the subject which is to be anobject are inputted from the image selector means, and, when havingdetected that all the items of image information have been inputted,causes collective inputting of all these items of abnormal patterndetection processing object image information concerning the samesubject for the same patient to the abnormal pattern detection processormeans. With this configuration, an automatic routing function isprovided which, from a number of and a variety of items of imageinformation inputted in the random order, automatically searches out andcollects the items of image information which provide a set for eachparticular patient, thus eliminating the need for manual operation bythe operator to select and output the items of image information to beoutputted to the abnormal pattern detection processor means.

However, none of the above-stated proposals for abnormal patterndetection processing provides any more than abnormal pattern detectionprocessing, output processing, and automatic routing processing, and iseffective for automatic detection of abnormal patterns, improvement ofthe observing and pattern reading performance for images of the affectedpart zones including an abnormal pattern which is an object ofassessment, or prevention of erroneous operation by the operator,delayed output of an abnormal pattern, etc. However, for automaticchecking for the specificity, the cancer detection rate, etc., enhancingthe pattern reading level, and providing contribution to improvement ofthe performance of assessment, there is a need to link the patternreading reports having a comment of the pattern readers about themammograms, etc. (hereafter a system to prepare a pattern reading iscalled a reporting system) to the algorithm for detection processing,etc., to provide a database for pattern reading reports, detectionresults, etc., which is well suited for statistical processing.

A first abnormal pattern detection processing method according to thepresent invention is an abnormal pattern detection processing methodwhich, on the basis of inputted image information, detects and processesan abnormal pattern in an image represented by the image information, inwhich, for each of a plurality of items of the image information, theresult of the detection processing is related to the corrected resultafter correcting the result and stored.

With this first abnormal pattern detection processing method, on thebasis of the stored plurality of results of detection processing andcorrected results, quantitative evaluation of the performance of thedetection processing can be performed.

A second abnormal pattern detection processing method according to thepresent invention is an abnormal pattern detection processing methodwhich, on the basis of inputted image information, detects and processesan abnormal pattern in an image represented by the image information, inwhich, for each of a plurality of pieces of the image information, theresult of the detection processing and the result of pattern readingassessment which has been obtained by pattern reading assessment usingthe image information are related to the result of pathologic assessmentconcerning the abnormal pattern and stored.

With this first abnormal pattern detection processing method, on thebasis of the stored plurality of results of pattern reading assessmentand results of pathologic assessment, quantitative evaluation of theperformance of the pattern reading assessment can be performed.

A first abnormal pattern detection processing system according to thepresent invention is a system realizing the above-stated first abnormalpattern detection processing method, i.e., an abnormal pattern detectionprocessing system which, on the basis of inputted image information,detects and processes an abnormal pattern in an image represented by theimage information, comprising memory means which, for each of aplurality of items of image information, relates the result of thedetection processing to the corrected result after correcting the resultand stores them.

It is preferable that this first abnormal pattern detection processingsystem further comprise evaluator means which reads out the plurality ofresults of detection processing and corrected results stored in thememory means, and on the basis of these results read out, performsquantitative evaluation of the performance of the detection processing.

A second abnormal pattern detection processing system according to thepresent invention is a system realizing the above-stated second abnormalpattern detection processing method, i.e., an abnormal pattern detectionprocessing system which, on the basis of inputted image information,detects and processes an abnormal pattern in an image represented by theimage information, comprising memory means which, for each of aplurality of items of image information, relates the result of thedetection processing and the result of pattern reading assessment whichhas been obtained by pattern reading assessment using the imageinformation to the result of pathologic assessment concerning theabnormal pattern and stores them.

It is preferable that this second abnormal pattern detection processingsystem further comprises evaluator means which reads out the pluralityof results of pattern reading assessment and results of pathologicassessment stored on the memory means, and on the basis of these resultsread out, performs quantitative evaluation of the performance of thepattern reading assessment.

The phrase “detects and processes an abnormal pattern in an imagerepresented by the image information” in the above-stated method andsystem refers to processing which detects an abnormal pattern in animage, and on the basis of the result of the detection, automaticallydetermines whether an abnormal pattern exists or not, and as disclosedin the above-mentioned Japanese Unexamined Patent Publication No. 8(1996)-294479 and Japanese Unexamined Patent Publication No. 8(1996)-287230, it refers to such processing as that which, formammograms, chest images, etc., automatically detects the existence ofcalcification (giving a style of abnormal pattern) suggesting theexistence of a breast cancer or other cancer and makes determination.

Here, the phrase “an abnormal pattern” means an image representing anabnormal condition of the subject, referring to, for the chest X-rayimage, mammogram, etc., for medical applications, for example, a patternshowing a symptom, such as those of tumor, calcification, thickening orpneumothorax of pleura, cancer, etc., which cannot be recognized with anormal pattern of a blood vessel or the like. In detecting andprocessing abnormal patterns, all of these abnormal patterns need not bedetected, but only the tumor pattern or only the calcification pattern,for example, may be detected and processed as an abnormal pattern, andtwo or more of these abnormal patterns may be detected and processed.For example, when the detection processing of abnormal patterns is byprocessing based on the algorithm which utilizes an iris filter todetect, as an abnormal pattern, an image portion where the image has ahigh concentration of density gradient (hereafter simply called irisfilter processing), the abnormal pattern is a tumor pattern, and whenthe detection processing of abnormal patterns is by processing based onthe algorithm for the morphology which detects, as an abnormal pattern,an image portion where the density varies in the range spatiallynarrower than the multiple structure element used by the detectionprocessing (hereafter simply called morphology filter processing), theabnormal pattern is a calcification pattern.

The image may include not only a true abnormal pattern, but also apattern similar to an abnormal pattern, in other words, an abnormalpattern candidate, which shows features similar to those of a tumor,calcification, etc., from the viewpoint of the image features of atumor, calcification, etc., and which is therefore not definitely anabnormal pattern, and requires final judgment by the pattern reader. Inthis specification, the phrase “detects and processes an abnormalpattern” refers to detection processing of not only a true abnormalpattern, but also such an abnormal pattern candidate.

The phrase “the result of the detection processing” means the result ofdetection processing the above-mentioned abnormal pattern, which mustinclude at least the result of automatic detection of the existence orabsence of an abnormal pattern or its candidate by the abnormal patterndetection processor means. It is more preferable that the result includenot only the result of detection of the existence or absence ofcalcification, etc., but also the contents of the items other than“existence of calcification” in the “detection result” in the BI-RADSreporting system later described.

The phrase “the corrected result after correcting the result” means theresult where the pattern reader, such as a radiologist, has carried outpattern reading assessment of an image (for example, an image of theaffected part zone including an abnormal pattern) outputted to the imageoutput means, and determined whether the result of the abnormal patternhaving been detected and processed is correct or not, and when theresult of the detection processing (the automatic detection processing)is not correct and the result of the detection processing has beencorrected, at least the result after the correction is included.

Here, the phrase “an image of the affected part zone including anabnormal pattern” means the vicinity including at least an abnormalpattern or an abnormal pattern candidate, and as the peripheral shape,an appropriate one of a variety of shapes, such as a rectangle, acircle, and an ellipse, can be adopted. The phrase “at least” in the “atleast an abnormal pattern or an abnormal pattern candidate” means thatpart of the detected abnormal pattern or the like may be included, i.e.,the entire abnormal pattern or the like need not always be included, andfurther the shape may be that created by tracing the periphery of theabnormal pattern or the like.

In outputting “an image of the affected part zone including an abnormalpattern” on the image output means, not only the abnormal pattern or thelike itself, but also an image of the vicinity zone is displayed. Inother words, it is preferable to output a combination of the abnormalpattern or the like with the original image. By doing this, the wholecan be easily conceived from the outputted image, and the abnormalpattern or the like can be easily positioned in the entire image.Particularly, assuming that the abnormal pattern detection processormeans outputs a combination of the abnormal pattern with the originalimage, these images can be outputted, being automatically laid out inaccordance with the predetermined layout, or the layout requirements areinputted from the terminal provided, being attached to the image outputdevice, and these images can be outputted, being laid out with thelayout requirements being met.

As the style of image layout, an appropriate one of the well-knownstyles, such as a style in which the original image and the abnormalpattern are each display-outputted on one display screen or one outputmedium in the form of multi-window; a style in which the original imageand the abnormal pattern are display-outputted, being overlaid one uponthe other in a single window; and a style in which a plurality of imageswhich are to provide a set (for example, right and left mammograms) aredisplay-outputted on one display screen or one output medium in the formof multi-window can be adopted. In creating the layout, it is morepreferable to adopt a style in which an image highlighted, enlarged orotherwise processed for the abnormal pattern is display-outputted, beingoverlaid on part of the entire image.

The phrase “the result of pathologic assessment concerning the abnormalpattern” means the result of the actual assessment of the abnormalpattern or the candidate area by, for example, palpatory audit or otherexamination by a clinician, affected part audit by use of a diagnosticneedle, a biological method, such as blood test, or the like.

The phrase “performs quantitative evaluation of the performance of thedetection processing” refers to finding an index which indicates thecertainty of the detection processing, particularly, a quantitativeindex which can contribute to improvement of the performance of thedetection processing (for example, making statistical processing) bycomparing the results of detection processing for a number of patientsand subjects with the corrected results.

The phrase “performs quantitative evaluation of the performance of thepattern reading assessment” refers to finding an index which indicatesthe certainty of the pattern reading assessment, particularly, aquantitative index which can contribute to improvement of the patternreading level (for example, making statistical processing) by comparingthe results of pattern reading assessment for a number of patients andsubjects with the results of pathologic assessment.

For the above-stated methods and systems, it is preferable that thedetection result, the corrected result, the result of pattern readingassessment, and the result of pathologic assessment be prepared andstored in conformity with BI-RADS (a US registered trademark). Here,“BI-RADS” (Breast Imaging Reporting And Data System) is the mammogrampattern reading standard system recommended by the ACR (American CollegeRadiology) which is intended to improve the performance of the mammogrampattern reading assessment, make the mammogram pattern readingobjective, and construct a data base, comprising the mammogram patternreading method (classification method), the reporting system, thefollow-up method (for pattern reading checking by comparing themammogram assessment with the biopsy), and the data base (NMD (NationalMammography Database)) creation method.

With the above-mentioned reporting system, it is recommended that, asthe reporting contents, entry be made about three items; PatientInformation, Breast Composition (Brief Description of Overview of EntireMammogram, Detection Result, and Summary), and Assessment Categories. Itis also recommended that comparison with past images (mammograms) beincluded whenever possible.

As the above-mentioned follow-up method, the method which compares themammogram assessment report (one of the categories 0 to 5 of theBI-RADS) with the result of the pathologic assessment by biopsy (theBIOPSY RESULT) to check (follow-up) the mammogram pattern reading isgiven. As a technique, by describing the reporting contents on the basisof the above-stated reporting system, the numbers of occurrences of TP(True Positive), FP (False Positive), FN (False Negative) and TN (TrueNegative), the Sensitivity, and the Specificity are found in accordancewith the specified formats (FORM A and FORM B), and thus the cancerdetection rate, the specificity, etc., can be known.

The above-stated database (NMD) creating method is the data baseconstruction method proposed by the ACR which is intended to be used forsuch applications as research with the use of the mammogram data,aftercare of the patient, and access to past images. Specifically, it isrecommended that the contents be recorded/stored about 122 items intotal, being classified into six data base record formats (detailsomitted). The NMD Data Collection which records only the critical itemsamong the 122 items is given.

With the above-mentioned reporting system, as the abnormal patterndetection processor means for detection processing of abnormal patternor its candidate, abnormal pattern detection processor means whichperforms iris filter processing to detect, as an abnormal pattern, animage portion where the image has a high concentration of densitygradient, abnormal pattern detection processor means which performsmorphology filter processing to detect, as an abnormal pattern, an imageportion where the density varies in the range spatially narrower thanthe multiple structure element used, or the like, as disclosed in theabove-mentioned Japanese Unexamined Patent Publication No. 8(1996)-294479, Japanese Unexamined Patent Publication No. 8(1996)-287230, or the like can be used.

The abnormal pattern detection processor means may be configured as adevice independent of the QAWS or constructed as a device well suitedfor constructing a network (refer to Japanese Unexamined PatentPublication No. 10 (1998)-233815). Specifically, as stated in JapaneseUnexamined Patent Publication No. 10 (1998)-233815), it is recommendedthat the abnormal pattern detection processing system comprise imageselector means which selects, among the items of image information whichare inputted from an image input device, being provided withsupplementary information which allows identification of the type ofsubject and the patient[[,]]the image information [concerning] aparticular type of subject which is to be an object of abnormal patterndetection processing by the abnormal pattern detection processormeans[[,]]and outputs it, and input monitor means which, when an item ofimage information concerning a subject which is to be an object ofabnormal pattern detection processing is inputted from the imageselector means, monitors that all the other items of image informationconcerning the same subject for the same patient which are to provide aset with the item of image information concerning the subject which isto be an object are inputted from the image selector means, and, whenhaving detected that all the items of image information have beeninputted, causes collective inputting of all these items of abnormalpattern detection processing object image information concerning thesame subject for the same patient to the abnormal pattern detectionprocessor means.

As the memory means, any type may be used so long as it can relate theresult of detection to the corrected result, or the result of detectionand the result of pattern reading assessment to the result of pathologicassessment and storing them. From the viewpoint of the necessity forstoring a number of items of patient information and creating a database, it is preferable to use a large capacity hard disk, an opticaldisk, or the like.

As the evaluator means, any type may be used so long as it can performquantitative evaluation of the performance of the detection processingon the basis of the results of detection processing and the correctedresults, or perform quantitative evaluation of the performance of thepattern reading assessment on the basis of the results of patternreading assessment and the results of pathologic assessment. It ispreferable to use a computer-aided type evaluator which comprises a CPUincorporating programs for making such quantitative evaluation, andperipheral circuitry. The evaluator means may, of course, be supplied asa memory medium, such as a CD-ROM, which records programs forquantitative evaluation.

The image input device includes a variety of image forming modalities,such as a CT device, an MRI device, and a CR device, and in addition tothese, a memory to store image information, etc. The image output deviceincludes image display devices (such as a CRT, and a liquid crystaldisplay), printers (including a laser printer (LP), such as a laserimager), etc. These image input device and image output device may beconnected to a network, such as a medical image network.

With the first abnormal pattern detection processing method and systemaccording to the present invention, for each of the plurality of itemsof image information (for example, for a number of patients), the resultof the detection processing is related to the corrected result aftercorrecting the result and stored, and therefore, by later comparing theresults of detection processing with the corrected results, thecertainty of the detection by automatic detection processing can befound by statistical processing, which allows improvement of thedetection processing algorithm, etc., to provide a higher specificityfor the automatic detection processing. If evaluator means forquantitative evaluation of the performance of the detection processingis provided, the above-mentioned statistical processing can beautomatically made for extreme convenience.

With the second abnormal pattern detection processing method and systemaccording to the present invention, for each of the items of imageinformation for a number of patients, the result of the detectionprocessing and the result of pattern reading assessment which has beenobtained by pattern reading assessment using the image information arerelated to the result of pathologic assessment concerning the abnormalpattern and stored, and therefore, by later comparing the results ofdetection processing and pattern reading assessment with the results ofpathologic assessment, the cancer detection rate, the specificity, etc.,can be found by statistical processing, which allows the performance ofcancer assessment by a doctor or an institution to be known, and thisresult to be utilized for providing a higher level of pattern reading.If evaluator means for quantitative evaluation of the performance of thedetection processing is provided, the above-mentioned statisticalprocessing can be carried out automatically for extreme convenience.

SUMMARY OF THE INVENTION

The purpose of the present invention is to offer an abnormal patterndetection processing method and system which can statistically processthe specificity and the cancer detection rate.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an embodiment (a style with networkconnection) of the abnormal pattern detection processing system of thepresent invention,

FIG. 2A and FIG. 2B are drawings giving examples of layout styledisplay-outputted to an image output device,

FIG. 3 is a chart giving the items in detail for the reporting contents,

FIG. 4 is a chart giving an example of pattern reading report,

FIG. 5 is a chart illustrating the method for giving a TP, FP, FN, or TNon the basis of the assessment category and the pathologic assessmentresult, and providing the formulas to find the sensitivity, thespecificity, and the value of PPV on the basis of the number ofoccurrences of TP, FP, FN, and TN,

FIG. 6 is a chart giving an example of format for statistical processing(FORM A),

FIG. 7 is a chart giving an example of format for statistical processing(FORM B),

FIG. 8 is a chart giving an example of result of processing on the basisof FORM A and FORM B,

FIG. 9 is a diagram illustrating an embodiment (a style without networkconnection) of the abnormal pattern detection processing system of thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinbelow, embodiments of the abnormal pattern detection processingmethod and system according to the present invention will be explainedwith reference to the drawings.

FIG. 1 is a diagram illustrating an embodiment of the system realizingthe abnormal pattern detection processing method according to thepresent invention using an example in which it is connected to a medicalimage network for assessment (hereafter simply called a network).

To a network 400 as shown in the figure, a CT device, an MRI device, aCR devices, etc., are connected as an image input device 430, and [[a]]CRT[[, a]] laser printer, etc., are connected as an image output device(image display means) 440. In addition, to this network 400 areconnected a QAWS 410 to which all the items of image informationinputted to the network 400 from the image input device 430 are inputtedand which stores and keeps all these items of image information forcollective control; a computer-aided image assessment device 420 whichperforms a variety of automatic assessments on the basis of the inputtedimage information; a reporting device to which a pattern reading reportrepresenting a comment as a result of reading an image outputted to theimage output device 440; and a server (large-capacity hard disk) 40 asmemory means which stores the result of detection processing in anabnormal pattern detection system 100 (later described, etc., andinformation (a pattern reading report) inputted from the reportingdevice 450, relating them to each other.

The abnormal pattern detection system 100 as shown in the figurecomprises image selector means 10, input monitor means 20, abnormalpattern detection processor means (abnormal pattern detection means) 30,a server 40, and evaluator means 50. The QAWS includes the imageselector means 10 constituting the abnormal pattern detection system100, and the computer-aided image assessment device 420 includes theinput monitor means 20, the abnormal pattern detection processor means30, the server 40, and the evaluator means 50 constituting the abnormalpattern detection system 100.

With the image selector means 10, among the items of image informationwhich are inputted from the image input device 430 to the QAWS 410 to bestored and kept, being provided with supplementary information whichallows identification of the type of subject and the patient, only theimage information concerning a particular type of subject which is to bean object of abnormal pattern detection processing by the abnormalpattern detection processor means is selected on the supplementaryinformation, and the selected image information is outputted to theinput monitor means 20 through the network 400, while the imageinformation other than the object image information is outputted to theimage output device 440 (such as a CRT display).

Here, “the image information concerning a particular type of subjectwhich is to be an object of abnormal pattern detection processing” meansthe radiation image information (regardless of whether the radiation istransmission radiation or self-emitting radiation) concerning a subject,particularly, the radiation image information concerning a subject forwhich a plurality of images are generally screened simultaneously(meaning “at one audit”) for a given patient. For example, it is imageinformation representing mammograms or chest images, etc., with which aplurality of images (for example, two or four) are generally screenedsimultaneously for the same patient.

“The supplementary information” means information representing the typeof subject (portion of a human body such as head, chest, abdominalcavity, breast, neck, and limbs), the screening position (the directiontoward the front, a side or the like), and the method of screening(simple screening, tomography screening, contrastradiogram screening orthe like), ID information for identifying the patient, informationindicating the date of screening, etc.

When an item of image information concerning a subject which is to be anobject of abnormal pattern detection processing is inputted from theimage selector means 10, the input monitor means 20 monitors that allthe other items of image information concerning the same subject for thesame patient which are to provide a set with the item of imageinformation concerning the subject which is to be an object are inputtedfrom the image selector means 10, and, when having detected that all theitems of image information have been inputted, causes collectiveinputting of all these inputted items of abnormal pattern detectionprocessing object image information concerning the same subject for thesame patient to the abnormal pattern detection processor means 30.

Here, “all the other items of image information concerning the samesubject for the same patient which are to provide a set with the imageinformation concerning the subject which is an object of abnormalpattern detection processing” are such that, when the image informationconcerning the subject is a mammogram, for example, two images, i.e., aplan image and a side image, are generally screened for each of theright and left breasts at one audit, thus, for one patient, four itemsof image information compose a set of items of information. Therefore,after the first one mammogram has been inputted to the input monitormeans, the remaining three mammograms correspond to “all the other itemsof image information”. This is also true for the other subjects. Forexample, when, for a chest image, image information from the directiontoward the front and image information from the direction toward a sideprovide a set, after one of these items of image information has beeninputted, the other of them corresponds to “all the other items of imageinformation”.

Whether or not all the items of image information concerning the samesubject for the same patient have been inputted to the input monitormeans 20 can be determined on the basis of the supplementary informationfor the inputted image information while referring to, for example, areference table or the like which is provided to store the number ofitems of information to be inputted for the same subject which ispreviously set depending upon the type of subject.

For mammography, for example, depending upon the contents of medicalexamination and treatment, image information only for either of theright and left breasts may be required (in such a case where the breaston one side has already been transected, and thus there is no need forscreening for that side), resulting in not all of the above-mentionedfour mammograms being required. In such case, if the input of the fourmammograms is continued to be monitored, no image information for themammograms which have not been taken will be inputted at all. Inaddition, there may occur a case where, for some cause, one or more ofthe items of image information which are to provide a set is notinputted at all. In these cases, if input of all the items of imageinformation which are to provide a set is awaited, there is thepossibility of an obstacle being caused to the subsequent processing,i.e., the processing of causing collective inputting of all the inputteditems of abnormal pattern detection processing object image informationconcerning the same subject for the same patient to the abnormal patterndetection processor means 30.

Then, it is recommended that the input monitor means 20 be provided witha processing style in which, when, within the previously set time fromthe moment at which the first item of image information for a particularpatient is inputted, the input of all the other items of imageinformation concerning the same subject as that represented by the firstitem of image information for the patient has not been detected, theinput of all the items of image information is regarded as detected, andonly the items of abnormal pattern detection processing object imageinformation which have been inputted are collectively inputted to theabnormal pattern detection processor means 30. By adopting such a style,the possibility of that the image information which is an object ofabnormal pattern detection processing cannot be inputted to the abnormalpattern detection processor means 30 at all can be eliminated.

On the basis of the image information inputted from the image inputdevice 430, the abnormal pattern detection processor means 30 detectsand processes the abnormal patterns (a tumor pattern or a minutecalcification pattern) in the radiation image represented by the imageinformation, and outputs the detected abnormal patterns to the imageoutput device 440, together with the original image, in the specifiedlayout.

Here, as an output layout of the abnormal patterns detected by theabnormal pattern detection processor means 30 and the original image, astyle in which images highlighted, enlarged or otherwise processed forthe abnormal patterns are display-outputted, and overlaid on part of theentire image is adopted, as shown in FIG. 2A or FIG. 2B. In other words,it provides a layout with which, in the right half of the output screenfor the image output device, the original image P for the right sidebreast and the enlarged views W1 to W4 of the affected part imagescorresponding to a plurality of abnormal pattern images P1, P11, P21,and P31, respectively, are displayed, while in the left half of theoutput screen, the original image P′ for the left side breast and theabnormal pattern images are displayed (the left half is partly omitted),which is a style in which, as shown in FIG. 2A, the detected abnormalpatterns are outputted, being overlaid on part of the entire image P,with the sizes of the enlarged views W1 to W4 of the affected partimages being set so that they are all equal to one another, or which isa style in which, as shown in FIG. 2B, the detected abnormal patternsare outputted, being overlaid on part of the entire image P, with thesizes of the enlarged views W1 to W4 of the affected part images beingset so that they correspond to the respective sizes of the detectedabnormal patterns.

The display output layout in this image output device 440 is not limitedto this, and a layout style in which the original image (for example,that of an entire breast) is displayed as an entire image, whileenlarged views of the affected part images comprising the abnormalpatterns and their surrounding area images are overlaid on part of theentire image, or a layout style in which, in the right half of thedisplay screen for the image output device, the original image of theright side breast and the abnormal pattern images are displayed, and, inthe left half of the display screen for the image output device, theoriginal image of the left side breast and the abnormal pattern imagesare displayed, with the abnormal pattern images being displayed,highlighted only in the entire image, or with the enlarged views of theaffected part images being overlaid on part of the entire image, orvarious other known styles as disclosed in Japanese Unexamined PatentPublication No. 8 (1996)-294479) can, of course, be adopted.

As the evaluator means 50, that of a computer-aided type is used whichincorporates a program for making quantitative evaluation of theperformance of the detection processing on the basis of the result ofautomatic detection processing by the abnormal pattern detectionprocessor means 30, and the result of correction of that result by thepattern reader, and a program for making quantitative evaluation of theperformance of the pattern reading assessment on the basis of the resultof pattern reading assessment by the pattern reader and the result ofpathologic assessment such as biopsy.

Next, the functions of the abnormal pattern detection system 100 of thepresent embodiment will be explained by using, as an example, a casewhere the image information which is an object of abnormal patterndetection processing is a mammogram.

First, from the image input device 430 (such as a CR device), items ofradiation image information representing many types of image areinputted to the image selector means 10 in the abnormal patterndetection system 100 through the network 400 in sequence, beingaccompanied by the respective items of supplementary information. Theimage selector means 10 reads the items of supplementary informationamong the items of image information inputted in sequence, and selectsthe mammograms, the subject information about which recorded in thesupplementary information is a breast, and outputs the selectedmammograms to the input monitor means 20. On the other hand, the imageinformation other than the mammograms is outputted to the image outputdevice 440 (such as a CRT display) requested for output through thenetwork 400.

When the first mammogram (image information toward the front for theright breast) is inputted from the image selector means 10, the inputmonitor means 20 temporarily stores the inputted mammogram in the memorymeans (not shown), reading the ID information for the patient among theitems of supplementary information for this mammogram.

For mammography, a set of four mammograms are generally taken, and so,the input monitor means 20 also reads the ID information for the othermammograms inputted in sequence from the image selector means 10 tomonitor that the other three mammograms (for example, image informationtoward the side of the right breast, image information toward the frontand image information toward the side of the left breast) for the samepatient which have been taken (are estimated to have been taken) as aset with the stored mammogram are inputted from the image selector means10. At the halfway stage, the other mammograms which are objects aretemporarily stored in the memory means.

When the input monitor means 20 detects that all the four mammogramshaving the respective items of information with the same ID informationhave been inputted, it inputs these four mammograms in a batch to theabnormal pattern detection processor means 30.

The abnormal pattern detection processor means 30 detects and processesthe abnormal patterns (a tumor pattern or a minute calcificationpattern) in sequence for the inputted four mammograms. Then, it arrangesthe detected abnormal patterns and the original images in the layout asshown in FIG. 2A or FIG. 2B to output them to the image output device440.

Through the above-stated series of functions, the image information isinputted to the network 400 from the image input device 430, and theabnormal patterns, etc., in the mammograms are displayed on the imageoutput device 440, the preparation for pattern reading being nowcompleted. Because this series of processing operations is automaticallyperformed without the need for manual operation by the operator,erroneous operation by the operator resulting from the complexity ofmanual operation, delay of output of the abnormal patterns caused by anerroneous operation, etc., can be prevented, and the abnormal patternscan be conveniently supplied for routine pattern reading.

After the preparation for pattern reading having been thus completed,the pattern reader, such as a radiologist, depresses the reportingpushbutton (not shown) provided on the reporting device 450 whileviewing the images display outputted to the image output device 440,then the entire network 400 is shifted to the reporting function, whichallows the desired information to be inputted to the reporting device450.

As the reporting contents (reporting function), records (descriptions)are given for the three items, i.e., patient information, breastcomposition (brief description of overview of entire mammogram,detection result, and summary), and assessment categories in conformitywith the above-stated BI-RADS reporting system proposed by the ACR. FIG.3 shows a chart giving the items in detail for the reporting contents.The specific method for recording the contents for these three items isas follows:

(1) The patient information is automatically stored in the server 40 bythe system 100 on the basis of the patient ID information, etc.,included in the above supplementary information.

(2) The result of detection processing of the abnormal patterns (theexistence of calcification) (the result of automatic detectionprocessing) by the abnormal pattern detection processor means 30 is alsoautomatically stored in the server 40 by the system 100, and when it isdetermined that abnormal pattern exists, the detected abnormal patternsare outputted to the image output device 440 in the layout as shown inFIG. 2 A or FIG. 2B together with the original image. In addition, notonly the result of detection processing of whether or not abnormalpattern i.e., calcification exists, but also the other detection resultitems as shown in FIG. 3 and the contents are automatically stored inthe server 40.

(3) The pattern reader (a radiologist or a clinician) makes patternreading assessment of the output image from the image output device 440to determine whether or not abnormal pattern exists, by referring to theresult of detection processing by the abnormal pattern detectionprocessor means 30 (as an assessment aid), and inputs the result. As theway of inputting the result, the displayed images which are determinedto be abnormal patterns by the pattern reader are circled (are clicked)by use of the ROI. Alternatively, by using the reporting device 450, thecorrect result of automatic detection processing is left as it is, andwhen the abnormal pattern detection processor means 30 gives anerroneous result of detection processing, it can be cancelled and whenthe abnormal pattern detection processor means 30 cannot detect anabnormal pattern, it can be added.

The reporting device 450 determines whether the result of automaticdetection processing by the abnormal pattern detection processor means30 is correct or not, by using the inputted pattern reading assessmentresult as a criteria (the correct result). For example, when an abnormalpattern is erroneously detected as if it existed, an FP (False Positive)is given. When an abnormal pattern which actually occurs is correctlydetected, a TP (True Positive) is given, and when an abnormal patternwhich actually occurs is erroneously detected as if it did not exist, anFN (False Negative) is given. Also, when existence of no abnormalpattern is correctly determined, i.e., when there is no wrong detection,a TN (True Negative) is given. This result of determination (which isequivalent to the corrected result after correcting the result ofautomatic detection processing) is stored in the server 40.

(4) The pattern reader carries out pattern reading assessment of theoutput image from the image output device 440 to input the assessmentcategory, the mammogram classification, and the comment in conformitywith the BI-RADS from the reporting device 450. These input data arestored in the server 40.

Thus, the pattern reading report of the patient information and theresult of detection processing, as well as the assessment category, themammogram classification, the comment, etc., as mentioned in the aboveparagraphs (1) to (4) (including the result of automatic detectionprocessing by the abnormal pattern detection processor means 30) isstored in the format in conformity with the reporting content items asshown in FIG. 3 which conform to the BI-RADS. When the pattern readingreport is stored in the server 40, it is converted into data which canbe used with each particular patient. FIG. 4 shows an example of apattern reading report.

(5) Besides this pattern reading report, the result of pathologicassessment (the result of final assessment) obtained from the actualassessment of the abnormal patterns or the candidate areas by thepalpatory audit by a clinician, the affected part audit by use of adiagnostic needle, the biological method, such as blood test, or thelike is also stored, being related to the pattern reading report.

Thus, by storing the information for a plurality of patients, a database is constructed. In constructing such a database, the NMD methodproposed by the ACR is adopted to record and store the contents of the122 items (the detail is omitted).

The evaluator means 50 automatically performs the following statisticalprocessing from the above-mentioned data base. In other words, from thedata which is based on the above paragraphs (2) and (3), the algorithmdetection rate and the algorithm erroneous detection rate for detectionprocessing by the abnormal pattern detection processor means 30 arestatistically calculated using the following formulas (in the followingformulas, TP and other symbols denote the respective numbers ofoccurrences).

Algorithm detection rate TP/(TP+FN)[%]

Algorithm erroneous detection rate FP/total number of images[occurrences per image]

By thus finding the algorithm detection rate, etc., for detectionprocessing, the performance of the algorithm for automatic detectionprocessing can be quantitatively evaluated. The algorithm can also beimproved based on this result.

From the data which is based on the individual assessment categories inthe above paragraph (4) and the individual results of pathologicassessment in (5), the specificity and the cancer detection rate arecalculated. In this case, because the reporting contents based on theBI-RADS reporting system are described, evaluation (statisticalprocessing) in conformity with the follow-up method of the BI-RADS canbe performed, and in this case, first from the data on the basis of theindividual assessment categories and the individual results ofpathologic assessment, TP, FP, FN, and TN are determined, and then inaccordance with the specified formats (FORM A and FORM B), thesensitivity, the specificity, etc., can be determined.

The method for giving TP, FP, FN, or TN on the basis of the 25assessment category and the result of pathologic assessment, and theformulas to find the sensitivity, the specificity, and the PPV (PositivePredictive Value) on the basis of the numbers of occurrences of TP, FP,FN, and TN are given in FIG. 5. FIG. 6 gives an example of FORM A, FIG.7 gives an example of FORM B, and FIG. 8 gives an example of result ofprocessing on the basis of FORM A and FORM B.

As shown in FIG. 5, with the follow-up method of the BI-RADS, theassessment category for the result of pattern reading assessment iscompared with the result of pathologic assessment by biopsy, and byusing the result of pathologic assessment as the criterion (the correctresult), it is determined whether or not the result of pattern readingassessment is correct. For example, in the case where the result ofpathologic assessment is “malignant”, when the category is 0, 4, or 5(which is equivalent to the result of pattern reading assessment beingthat an abnormal pattern exists, or the result being uncertain), TP isgiven, and when the category is 1, 2, or 3 (which is equivalent to theresult of pattern reading assessment being that no abnormal patternexists), FN is given, while, in the case where the result of pathologicassessment is “benign” or no cancer has been discovered within the lastone year, when the category is 0, 4, or 5, FP is given, and when thecategory is 1, 2, or 3, TN is given. The sensitivity, the specificity,and the PPV can be found as follows on the basis of the numbers ofoccurrences of TP, FP, FN, and TN (in the following formulas, TP andother symbols denote the respective numbers of occurrences).Sensitivity=TP/(TP+FN)Specificity=TN/(TN+FP)PPV=TP/(TP+FP)

For the follow-up method of the BI-RADS, a technique which uses thenumbers of occurrences of TP and others, and the DATA ITEM forclassification of them to perform more detailed statistical processingas shown in FIG. 6 and FIG. 7 is provided. With this technique, theindexes other than the above-mentioned sensitivity, etc., (for example,cancer detection rate) can be obtained.

By thus finding the sensitivity, the specificity, etc., the performanceof cancer assessment by a doctor or an institution can be known toobtain a guideline to a higher level of pattern reading.

The abnormal pattern detection processing system of the above-statedembodiment has been explained on the assumption that it is configured asa system connected to the network. However, it can, of course, beconstructed as an independent system which is not connected to thenetwork. FIG. 9 is a block diagram illustrating an embodiment in whichthe above-stated abnormal pattern detection processing system 100 isconstructed as an independent system. The abnormal pattern detectionsystem 100 as shown in FIG. 9, as with the above-statednetwork-connected system, comprises image selector means 10, inputmonitor means 20, abnormal pattern detection processor means 30, aserver 40, and evaluator means 50. An image input device 430 isconnected to the image selector means 10, and an image output device 440is connected to the image selector means 10 and the abnormal patterndetection processor means 30. A pattern reading report from a reportingdevice 450 is inputted to the server 40, as well as the result ofdetection processing by the abnormal pattern detection processor means30, these being configured so as to be connected to the server 40. Thefunctions of this abnormal pattern detection system 100 are the same asthose of the above-stated system which is connected to the network, thusdetailed explanation of them is omitted.

With the abnormal pattern detection processing system of theabove-stated embodiment, it is assumed that the image selector meansselects only the mammograms to output them to the input monitor means,but with the abnormal pattern detection processing system of the presentinvention, the image information to be selected is not limited tomammograms, and various types of image information, such as chest imageinformation, can be selected, provided that the image information to beselected is an object of abnormal pattern detection processing, and itis made up of a plurality of items of image information which aredifferent from each other because they differ in the direction ofscreening, etc., but which form one set screened for the same subject.

The input monitor means may be provided with means having a functionwith which, when, within the previously set time from the moment atwhich the first item of image information for a particular patient isinputted, the input of all the other items of image informationconcerning the same subject as that represented by the first item ofimage information for the patient has not been detected, the input ofall the items of image information is regarded as detected[[,]]and onlythe items of abnormal pattern detection processing object imageinformation which have been inputted are collectively inputted to theabnormal pattern detection processor means, so that the likelihood of itnever being possible to input the image information which is an objectof abnormal pattern detection processing to the abnormal patterndetection processor means, when one or more of the items of imageinformation to provide a set is never inputted for some reason, can beeliminated.

The abnormal pattern detection processing system of the above-statedembodiment has been explained on the assumption that, when the abnormalpattern detection processor means 30 performs detection processing anddetermines that an abnormal pattern exists, an image of the affectedpart zone including the abnormal pattern is outputted to the imageoutput device. However, the image of the affected part zone need notalways be displayed so long as the abnormal pattern in the imagerepresented by the image information is detected and processed on thebasis of the inputted image information.

The above-stated embodiment has been explained on the assumption thatthe abnormal pattern detection processing method and system according tothe present invention is used for medical assessment application.However, the scope of application for the present invention is notlimited to medical assessment application, but can also be applied toradiation assessment devices for non-destructive inspection.

As described above in detail, with the abnormal pattern detectionprocessing method and system according to the present invention, bycombining the reporting system with the abnormal pattern detectionprocessing system and relating the result of detection by the abnormalpattern detection processing system to the pattern reading report, etc.,the pattern reader (such as a doctor) can check for the performance ofassessment (the specificity) and the sensitivity, use the patternreading report for improvement of the detection processing algorithm,and as a result, can perform pattern reading at a higher level andcontribute to improvement of the performance of assessment.

1.-2. (canceled)
 3. An abnormal pattern detection processing methodcomprising: detecting an abnormal pattern in an image, based on inputtedimage information; processing the detected abnormal pattern; performinga pattern reading assessment using the image information; performing apathologic assessment of the abnormal pattern; wherein the pathologicassessment comprises at least one of palpatory audit, examination byclinician, audit by diagnostic needle, and a biological analysis;relating a result of the detected abnormal pattern processing and aresult of the pattern reading assessment to a result of the pathologicassessment, for each of a plurality of items of the inputted imageinformation, for each patient; and storing the plurality of processeddetected abnormal pattern results, the plurality of pattern readingassessment results and the plurality of pathologic assessment resultsfor each patient.
 4. An abnormal pattern detection processing methodaccording to claim 3, wherein quantitative evaluation of the patternreading assessment is performed, on the basis of said stored pluralityof pattern reading assessment results and said stored plurality ofpathologic assessment results.
 5. An abnormal pattern detectionprocessing system which detects and processes an abnormal pattern in animage represented by image information on the basis of inputted imageinformation, comprising: a means relating a result of said detectionprocessing to a corrected detection processing result, for each of aplurality of items of said image information and storing the pluralityof detection processing results and the plurality of corrected detectionprocessing results for each patient; and evaluator means for performingquantitative evaluation of the detection processing on the basis of saidplurality of results of detection processing and corrected detectionprocessing results stored in said relating and storing means. 6.(canceled)
 7. An abnormal pattern detection processing system, whichdetects and processes an abnormal pattern in an image represented byimage information on the basis of inputted image information,comprising: a means relating a result of said detection processing and aresult of a pattern reading assessment using said image information to aresult of pathologic assessment concerning said abnormal pattern,wherein the pathologic assessment comprises at least one of palpatoryaudit, examination by clinician, audit by diagnostic needle, and abiological analysis for each of a plurality of items of said imageinformation for each patient, and storing the plurality of detectionprocessing results, the plurality of pattern reading assessment resultsand the plurality of pathologic assessment results for each patient; andevaluator means for performing quantitative evaluation of the detectionprocessing on the basis of said plurality of results of detectionprocessing and corrected detection processing results stored in saidrelating and storing means.
 8. (canceled)
 9. An abnormal patterndetection processing method according to claim 3, wherein the processingautomatically determines whether the abnormal pattern exists or not,based on a result of the detection of said abnormal pattern in saidimage.
 10. An abnormal pattern detection processing system according toclaim 5, wherein the processing automatically determines whether theabnormal pattern exists or not, based on a result of the detection ofsaid abnormal pattern in said image.
 11. An abnormal pattern detectionprocessing system according to claim 7, wherein the processingautomatically determines whether the abnormal pattern exists or not,based on a result of the detection of said abnormal pattern in saidimage.